








; ■ 

i . • • 

I REPORT RCSUHES 

I ED 017 619 vT 000 Q4 e 

NORTH ATLANTIC RE610NAL RESEARCH CONFERENCE IN AGRICULTURAL 
EDUCATION! CONFERENCE PROCEEDINGS (ITHACA i NOVEMBER 5-7, 

I ises). 

STATE UNIV. OF N.Y., ITHACA 

PUB DATE 63 

eORS PRICE MF-$0.25 HC-S2.08 SOP. 

DESCRIPTORS- ^AGRICULTURAL EDUCATION, ^EDUCATIONAL RESEARCH, 
♦research METHODOLOGY, «OFF FARM AGRICULTURAL OCCUPATIONS, 
SPEECHES! «CONFERENCES, NORTH ATLANTIC REGION, 



THIS PUBLICATION REPORTS ON SIGNIFICANT SPEECHES, 
CURRENT RESEARCH, AND COMMITTEE WORK DURING THE 3-DAY 
CONFERENCE FOR TEACHER EDUCATORS, SUPERVISORY STAFF MEMBERS, 
TEACHERS OF AGRICULTURE, AND GRADUATE STUDENTS IN 
AGRICULTURAL EDUCATION. DIGESTS OF THE FOLLOWING SPEECHES ARE 
GIVEN-- (1) "PROGRESS REPORT OF STATE STUDIES IN NON-FARM 
AGRICULTURAL OCCUPATIONS," BY ROBERT E. TAYLOR, (2) 

"DESIGNING RESEARCH IN EDUCATION" AND "SUGGESTIONS FOR THE 
PREPARATION OF EDUCATIONAL RESEARCH MATERIALS," BY DAVID R. 
KRATHWOHL, AND (3) "RESEARCH IMPLICATIONS OF THE STUDY OF 
EMPLOYMENT OPPORTUNITIES AND NEEDED COMPETENCIES IN NON-FARM 
AGRICULTURAL OCCUPATIONS," BY DUANE M. NIELSEN. COMPLETE 
TEXTS ARE GIVEN FOR (1) "RESEARCH EXPERIENCES," BY LAWRENCE 
B. DARRAH, AND (2) "OUR NATIONAL CENTER FOR ADVANCED STUDY 
AND RESEARCH IN AGRICULTURAL EDUCATION," BY C. W. HILL. 
REPORTS ARE GIVEN FOR THE FIVE WORK GROUPS WHO STUDIED 
RESEARCH PROJECTS OR PROPOSALS SUBMITTED IN ADVANCE BY 
SEVERAL STATES AND SUGGESTED IMPROVEMENTS IN THEIR DESIGN. 
(PS) 








i- ; ; 

ill: 



DOCUMENT HUIED FROM BKT AVMI^ 



> -i: 












in 







Agricultural Education Di^ 
Rural Educarion Departs 
York State 

i&aca, New 









1 



U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE 
OFFICE OF EDUCATION 



THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION 
POSITION OR POLICY. 

NOFTH ATLANTIC REGIONAL RESEARCH 
CONFERENCE IN AGRICULTURAL EDUCATION 



Teacher Educators, Supervisors, Program Consultants, Guests, and 
Graduate Students of Vocational Education in Agriculture from: 



Cornell University 

Delaware State Education Department 

Michigan State University 

New York State Education Department 

Pennsylvania State University 

Pennsylvania State Education Department 

Rutgers University 

The Ohio State University 

University of Delaware 

University of Iviaryland 

University of Massachusetts 

University of N ew Hampshire 

University of Vermont 

U, S, Office of Education 

West Virginia University 



New York State College of Agriculture 



Cornell University 
Ithaca, New York 



November 5, 6 & 7, 1983 



1 

















PPOGPAM FLAKNIMG COMAOTTEE 

Harold P, Cushman, New York, Chairman 
Glenn Z, Stevens, Pennsylvania, Pesearch Committee 
Clodus P, Smith, l^.'Iaryland, Pesearch Committee 
V/illiam H, Annis, i'Tev/ Hampshire, Professional Information Com, 
William E, Drake, New York, Professional Pelations Committee 
Bruce A, Gaylord, Vermont, Teacher Education Committee 
Palph C, S, Sutliff, New York, Standards and Policies Committee 
Joe P, Bail, New York, Cornell University 
Duane M, .Nielsen, U, S, Office of Education, Pesearch and 
Teacher Education Specialist 

H, N, Hunsicker, U, S, Office of Education, Program Specialist 



CONFEPENCE SECPETAPY 

William E, Drake, Assistant Professor, Agricultural Education, 
Cornell University 



CONFEPENCE CHAIPMAN 

Harold P, Cushman, Associate Professor, Agricultural Education, 
Cornell University 



• • 

11 






North Atlantic Pegional Research Conference 



Preface 



This publication reports significant speeches, current research, 
and committee work conducted during the 1963 Annual Research Conference 
for Teacher Educators, Supervisory Staff Hembers, Teachers of Agricul- 
ture and Graduate Students in Agricultural Education from the North 
Atlantic Region. 

The conference proceedings are organized in five sections, each 
reporting one of the five general sessions conducted during the three-day 
conference. The first section includes a 'Progress Report of State 
Studies in Non- Farm Agricultural Occupations '■ by Robert E. Taylor. . 

This section also includes reports of research in off-farm agricultural 
occupations presently being conducted in certain states of the North 
Atlantic Region. 

The second and third sections report the general sessions of the 
conference devoted to research design. These sections include a digest 
of the presentation entitled “Designing Research in Education” by David 
R, Krathwohl. Also included in this section is Dr. Krathwohl's 
"Suggestions for the Preparation of Educational Research Proposals, " 

Section four presents the research proposals which were the 
subjects of study for the various work groups. This section also lists 
the membership of the respective workshop groups. 

Critiques of research proposals are reported in section five, 
which includes the work group committee reports. 

The last section of the proceedings is devoted to speeches by 
L. B, Darrah, C, W, Kill and Duane M, Nielsen pertaining to research 
experiences, the National Center for Advanced Study and Research, 
and implications of the non-farm agricultural occupations studies, 
respectively. 

The conference proceedings should prove to be a valuable 
reference in planning meaningful research and v/riting research pro- 
posals to be submitted for funding. The information included herein 
is relevant, recent and provocative in its application to the current 
research needs in agricultural education. 



v/illiam S. Drake 
Conference Secretary 
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CQNFEBENCE PP0C5EDIMGS 



First General Session - November 5, 1963 

Chairman! Jo6 Bail Secrstaryt William H, Annis 

The North Atlantic Pegional Fesearch Conference was called to 
order at 9 a. m. by Joe P. Bail of New York. William H. Annis of 
New Hampshire served as secretary for the first day of the conference. 

Dr. Marvin D. Glock, Acting Dean, School of Education, Cornell 
University, welcomed the group to Cornell. 

Dr. Harold B. Cushman, Program Chairman, reviewed the con- 
ference program with the group. He stated that the conference theme was 
to be research concerning non-farm agricultux al occupations. 



FBOGBESS BEFOBT OF STATE STUDIES IN NON-FABM 
AGBICULTUBAL OCCUPATIONS 

Bobert E. Taylor, Director 
The National Center for Advanced Study and Besearch 

. • • 

Digest of Presentation 

• • 

• L Status of Agricultural Occupations Studies - United States 

A. NIassachusetts, New Jersey, North Dakota and Vermont hav’ 
reported no plans. 

B. Alabama, Arizona, Florida, Louisiana, Michigan, Minnesota, 
Mississippi, Montana, New Hampshire, New Mexico, Tennessee, 
Texas and Utah have studies planned. 

C. The following states have studies underway: California, Georgia, 
Indiana, Iowa, Kentucl^y (2)'", Mchigan, Missouri, Nebraska, 

North Carolina, Ohio, Oklahoma, Pennsylvania, South Carolina, 
Virginia, Washington, and West Virginia. 

D. Studies have been completed in Colorado, Illinois, Indiana, ICansas, 
NIaryland (2), ‘Michigan (2), and North Carolina (3). 

E; Beports are available of the previous completed studies with the 

Exception of Illinois and Kansas. 

F. Thirty-two states are actively involved in a total of 45 studies. 



* The numbers in parentheses indicate the number of studies in that par 
ticular state. 
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n. Person Conducting the Studies 




A, Primary responsibility 

1. State Supervisory Staff 

2. Teacher Education Staff 

3. Joint Staffs 

4. Graduate Students 

5. Others with some duplication, 



3 

9 

9 

13 

10 






B. Cooperators include 

1. Colleges of Agriculture 

2. Experiment Stations 

3. State unemployment compensation commissions 

4. Area schools 

5. Local districts 

6. industrial education centers 

7. State-wide agricultural conferences 

8. Teachers of vocational agriciilture 

9. State departments of education 

10. Otter vocational services 

• 

TTT. How Studies Are Being Financed 



o 

ERIC 



A, Primary soiirces 

1. State board for vocational education. ••••15 

2. University ^ 

3. Graduate Students • • • ’ 

4. Others 

B. Others sharing in financing 

1. Colleges of Agriculture 

2. Experiment Stations 

3. State unemployment compensation commissions 

4. Title m and VIII, N. D. E. A. 

5. Area schools - industrial education centers - junior colleges 

6. Local districts \ 

7. Graduate assistantships 

IV. Phases of the Program for VMch Studies Have Major Implications 

A, The phases of the program primarily affected include: 

1. High school 

2. Young farmer. • • ^ 

3. Adult farmer. " 

4. Agriciiltural technician training 13 

5. Post high school ^ 

B. Others, including area schools and miscellaneous. 5 









V. Sources of Data 

A. Pespondents 






•M 



1. Employers * • • 

2. Employees VV*!* 

3. Others, including jury of experts and former students 



B. How selected 

1. Pandom (approximately one-half) 

2. Entire student universe (limited geographical area or based 
on known agricultural businesses) 



C, Geographical areas included 

1. State 

2. Economic areas (several counties) 

3. County 

4. School district 



VI. How Will Data Be Collected 

A. IVIail surveys 

B. Interviews 

C. Other 



Vn. Data Available from the Studies 



A. Identifcation of occupations for which knowledge of agriculture 
is desirable 



B. Present number of employees in above categories 2J 

C. Annual turnover and entry opportunities in these occupations. . 23 



D. Beginning salary levels 

E. General requirements, agricultural, and olher^competencies 
* needed for entry and advancement 



1. Level of competency (e, g. , semi-skilled, skilled, 

technical) 

2. Preferred minimum formal education 

3. Preferred minimum experience 

4. Minimum age for entry 

5. Preferred residential background 

6. Labor laws and union regulations 

7. Licensing and certification 



24 

25 
20 
18 
14 
12 

8 







Tndividual State Bepoits 



BeDorter; Claude McGhee, Assistant Professor, Agricultural 
Education, West Virginia University. "Non-Farm Agricultur^ Occupations 
for Which Training May Be Provided by Vocational Education in Agriculture. 
The. report was. as follow^ • .• 



* The aini of the study is to determine the scope and extent of non- 
farm employment in agriculture and to identify some occupations or 
occupational areas for which training programs could be implemented on 
the high school and post-high school levels which would provide skills 
contributing to successful entrance into certain of these occupations. 



Specific Objectives : 

1, To conduct a census of firms, businesses, industries, agencies, and 

organizations with employees who are required to possess agricul- 
tural competencies, 

* * * ; • , 

2, To interview these firms., businesses y etc, , to determine the number 
of employees .in various job classifications as well as some of the 
basic requirements for entering each occupation. 

3, To identify some occupations or occupational areas for wnich training 

pro.grams could, be, implemented. on high school and post -high school 
levels which would contribute to successful entrance into certain of 
these occupaticns 



Universe of Study: . . . 



The total occupational spectrum of the State constitutes the universe 
of the study. This is desirable because errors of sampling can be avoided 
and useful data will be provided at both the state and local levels. This 
imiverse.was possible, because*- .... 



1, T^e sparseness of agricultural industries and the limited number 
* of non-farm agricultural occupations enable a comprehensive stucfy, 

(a) State population 

(b) Limited number of commercial farms 12, 609 

(c) Estimated number of non-farm agricultural occup, 15, 000 



2, Teachers of vocational agriculture are available and are willing to 
assist in the collection of the necessary data. 



Definition: 



Acyric^^lti^T^^l Occupation: An occupation in which the worker needs 
competencies in .o.ne pr.mpr© of the primary areas of plant science, animal 
science.^ soil, science,, agricultural. mechanization, and agricultural business 
mana.geme.nt 



% 
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Assumptions: 



1, Teachers of vocational agriculture v/ill malce valid identifications of the 
non-farm agricultural occupations in their respective counties, 

2, Data collected by interviews conducted by the 116 teachers of voca- 
tional educators, after suitable instruction, will be uniformly valid, 

3, The identity of the non-farm agricultural occupations is basic to 
vocational agriculture. 



Procedures: 



1, An instrument was prepared to structure the collection of data by 
interview, 

2, Teachers of vocational agriculture and other agricultural educators 
will compile a listing of all firms, businesses, industries, agencies, 
organizations, and persons to be interviewed, 

3, Data will be collected by interviews v/ith employer representative 

* having responsibility for the personnel of the business, 

4, Teachers of vocational agriculture v/ill serve as interviewers in 46 
of the 55 counties. Arrangements have been made for some teachers 
of sparsely populated areas to assist in the more populous- area^, a 
teacher of vocational agriculture in each of the 46 counties has been 
designated as county chairman to coordinate the county study, 

(a) Nine district training sessions on the principles, procedures, and 
methods of interviev/ing were held prior to the collecdon of data, 

5, Data from the nine counties not served by wocational agriculture will 
be gathered by the supervisory and teacher training staffs and mid- 
year graduates in agricultural education. 



Time and Event Sequence Schedule of the Study: 



Order of Events 


Date 


Approval of study and development of preliminary 
plans by State staff 


Completed 


Preliminary approach to teachers of vocational 
agriculture concerning development of study 


Completed 


Development of interview instrument and handbook 
for interviewing 


Completed 


Definition and description of study v/ith teachers at 
Annual Vo-Ag Teachers Conference 


August 8 
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Order of Events 

5, Workshops for training teacher -interview personnel 

6, Pilot interview program - 3 counties 

7, Period of statewide surve^^ 



Date 

September 12-23 

October 1-15 

November- 1 - 
January 31 



8. Development of procedure for organizing, 

tabulating, and analyzing data November 



9, Computation of machine tabulation of data March 1 

10. Preparation of research report ]\,iarch - April 

11, Completion of report 

iimmi 

% 

Reporter: David P. McClay, Head, Department of Agricultural 
Education, Pennsylvania State University, '‘Technical Education Needs of 
Persons Engaged in Agricultural Occupations, " The report appears below: 



Obiectives 

1, To identify agricultural occupations and job titles in Pennsylvania 
and to estimate present numbers of employees and annual entry 
opportunities, 

2, To list competencies needed for entry and advancement and to deter- 
mine job characteristics such as salary, minimum age, labor law and 
union restrictions, required education and experience, licensing and 
certification. 

3, To group occupations and job titles for which there are common 

technical ediication needs, - - . . 

Peasons for Undertaking the “Work: 

Changes in the agricultural industry during the past decade have 
resulted in new opportunities for employment in a wide range of agricultural 
occupations. The occupational areas, the number of persons employed in 
them, and the agricultural competencies needed should be determined -in order 
to provide guidance in establishment of technical education programs. 

Education for A Chancrina World of Work (4), the summary report of 
the Panel of Consultants on Vocational Education, points to the need for voca- 
tional and technical education for some 80 per cent of our youth during the 
present decade. The report states, "The vocational agriculture program, 
under Federal reimbursement, should be broadened to include instruction 
and increased emphasis on management, finance, farm mechanization. 




conservation, forestry, transportation, processing, marketing the products 
of the farm, and other similar topics. To attain the broad objectives 
stated by the Panel of Consultants on Vocational Education, occupations and 
specific job titles in the agricultural industry must be identified. 

I The needs for knowledge, skill, and the ability to make decisions 

f become the basis for the subj ect matter content of technical education 

I courses that enroll persons preparing for or engaged in each of the agri- 

cultural occupation groups. Competencies specific to a job title may be 
taught individually in a supervised cooperative education (on-the-job 
training) program. 

The attainment of the objectives in this study should facilitiate 
(1) thev/riting of job descriptions, (2) the clustering of jobs into training 
categories, (3) the structuring of course outlines and curricula, and 
(4) the testing of experimental programs in a search for administratively 
feasible ways of offering the courses. 

Previous Work and Present Outlook : 

Agricultural occupation studies have been completed in North 
Carolina (1), Washington (2), California (6, 7), and New York (8), 

Cameron (3) surveyed Huntingdon County in Pennsylvania. The studies 
show increasing employment opportunities in agricultural occupations other 
than farming. They also indicate the importance of some farm experience 
for success in these occupations and that there are minimums of agricultural 
competence for entry. Many agricultural jobs require post-high school 
technical education. There is demand for specialized knowledge in agricul- 
tural science, in business, and in mechanics. 

A Pesearch Coordination Conference on Agricultural Occupations (6) 
developed interview schedules based on studies in progress in Illinois, 
Kansas, Ohio and New York. Pevised versions of the schedules have aided 
in the design of this investigation. 

Literature Cited 

1. Blackmon, J. H. , Dav/son, C. G. Need for Training for Mon- Farming 

. Agricultural Occupations. Department of Public Instruction, Paleigh, 

North Carolina. 1961. 

2. Brown, B. L. Training Needs of V/orkers in Business Associated v/ith 

» Agriculture. State Board for Vocational Education, Olympia, Washington. 

1959. 

3. Cameron, li. L. Present and Potential Off-Farm Agricultural Occupa- 
tions and Training Needed for These Occupations by Vocational Agriculture 
Graduates in Huntingdon County, Pennsylvania. M. Ed. Pesearch Study, 
Library, The Pennsylvania State University. 1932. 

4. Education for A Changing World of Work. OE -80021. Washington: 
Government Printing Office. 1963. 

5. Halterman, J. J. Technicians in Agriculture. State Department of 
Education, Sacramento, California. 1962. 
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6, Report of Research Coordination Conference on Agricultural Occupataons. 

* The Ohio State University, Coluinbus, Ohio. 1963, 

7, Sutherland, S. S., Thompson, O, E, Training Required ^ Workers m 
Agricultural Business and Industry. State Department of Education, 
Sacramento, California. 1957. 

8, Tom, F. K, T, , Hill, C, W, , Greene, IC. L, Employment Opportumties 
in Certain Occupations Belated to Farming in the Syracuse, Nev/ York, 
Economic Area, Bural Education Department, Cornell University, 

Ithaca, New York, 1961, 



Procedure: 

Agricultural occupations may be defined as those in which workers 
need competencies in one or more of the primary areas of plant science, 
animal science, agricultural business management and marketing, and^ 
agricultural mechanization. Needs for technical education are expected 
to emerge from the developmental sequence of the study design as follows: 

1, Lists of businesses and services will be obtained from the Bureau of 
Employment Security, Department of Labor, and from the Division of 
Sales and Use Tax, Department of Bevenue, Persons in government 
and in orofessions are on available lists. The Agricultural Stablization 
and Conservation Service has the names of farmers. The Standard 
Industrial Classification Llanual will be used to aid a comnaittee of ^ 
business and professional leaders in each selected county in establishing 
random samples of owners and managers to be interviewed, 

2, Two interview schedules, pre-tested in a pilot study of Snyder County 
in 1963, will be used. Form I obtains general information about the 
business or service and lists levels of employment and spec iiic job 
titlss for 6mploy66s nooding agricultural compotoncios# Form II 
provides for detaUed job characteristics and a checking of structured 
lists of specific competencies, 

3, The data schedules are designed to be summarized in ways that will 
* discover technical education needs common to groups of occupations 

^d job titles, thus contributing to efficient scheduling of courses and 
utilization of staff and facilities. 



Probable Duration: 
Three years. 



Financial Support: 



Estimated annual expense 

State, Smith-Hughes and George Barden Fimds 
Salaries 
I^^intenance 



$7362 
, 550 



Dept, of Public Instruction 
NDEA, Title VIH funds 

Salaries $3136 

Maintenance 3914 

Personnel : 

N. K. Hoover, D. B, McClay, G. Z. Stevens 
Advisory Committee : 

H, J, Bonser, P. C, Bealer 
Institutional Units Involved : 

Department of Agricultural Education 
Cooperation: 

Commonwealth of Pennsylvania, 

Department of Public Instruction 

Approved by Pennsylvania Agricultural Experiment Station: 

M, A, Farrell, Director November, 1963 



Feporter : Joe P. Bail, Chairman, Agricultural Education Division, 
Cornell University. "Progress Peport on Surveys in New York State to 
Determine Employment Opportunities and Competencies Needed in Agricul- 
tural Occupations Off -the- Farm. " The report is as follows: 



Mr, Chairman, and Members of the North i\tlantic Pegional Pesearch 
Conference in Agricultural Education; We have been asked to share our ex- 
perience with you on this subject. As many of you are av/are, this particular 
subject is of paramount importance to us at this time in agricultural education, 
v/ith the expansion of vocational and technical education in the offing, it is 
imperative that we become better informed of the need for trained workers 
in agriculture. Since the Census of Agriculture gives a rather comprehensive 
report on farming, we are assuming that this data (i. e. , Census Data) will 
suffice for determining employment opportunities in the basic part of 
agriculture - farming. However, as you well know, facts and figures re- 
garding employment opportunities in agricultural occupations off-the-farm 
are largely guestimates - varying from 15 to 40v5 of our total work force, 

A recent example of this is the report in the September -October issue of 
B etter Crops with Plant Food w hich cites the following employment figures: 



Farm Production 
Farm Production Supply Service 
Processing of Agricultural Products 
Wholesaling of Agricultural Products 
Petailing of Agricultural Products 

TOTAL 



- 3, 700, 000 farmers 

- 5, 600, 000 employees 

- 3, 000, 000 e mployees 

- 1, 000, 000 employees 

- 3, 000, 000 employees 
-16, 300, 000 
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I'm not quarreling v/ith these figures - perhaps they are too low or 
maybe too high - but I'm sure that it would be difficult to substantiate many 
of them. This then, seems to me, to be a primary purpose of our surveys, 
"to secure more factual figures regarding employment in agricultural 
occupations, particularly those not concerned with actual production of 
agricultural products. • 

Let me review with you the kinds of information and the methods in 
which we hope to secure it. These activities are being carried out by 
members of our Joint Staff, by teachers of agriculture, and by others. 

Kinds of Information. By now the form being used to collect some 
of these data is well known to you. It has been labeled Part A or Form A 
in most cases. Basically, it calls for the collection of data from any 
company, business, agency, or organization which is engaged in any way 
with an agricultural product from the time it leaves the producer to the 
time it gets to the consumer, including all the related services along the 
way. In addition, it calls for a listing of the total number of employees 
needing competencies in agriculture plus a breakdown of titles and a 
classification of the level of employment. 

The second type of information relates to the actual competencies 
needed by specific occupational titles. This has generally been labeled 
Form B, This section, I'm sure, will vary considerably from one part of 
the country to another, although there will still be many common elements 
in it. At the present time our efforts are primarily focused on Form A - 
Employment Opportunities. However, ive are using Form B in some pilot 
surveys in order to refine it. 

Methods of Securincr Information. Since our state is large in geo- 
graphic area as well as population and number of businesses, we have had 
to make certain decisions regarding the scope and method of collecting 
data. In general, our survey cannot be called a state-wide one. We will 
be contacting some agencies, businesses, etc. , on a state- wide basis but 
a complete state survey would not, in our opinion, be feasible. Let me 
indicate to you then our manner of approaching this problem. 

1, Some surveys will be conducted on a school district basis in which a 
complete census will be carried out. Two such surveys have been 
comple^'ed with at least two others in the planning stage, 

2, Some surveys will be conducted on a county basis, either a complete 

census or a sampling. One such study is complete with two others 
underway, " 

3, Some surveys of specialized businesses or concerns will be made. One 
such survey of agricultural machinery and major farm equipment dealers 
is practically complete. Another regarding employment opportunities in 
ornamental horticulture, greenhouses, etc. , is planned. 



4, Selected surveys of stat3-;yide businesses, concerns, or agencies will 
be made. These include agricultural cooperatives, governmental 
ag encies, association lists, etc, A somev/hat modified Form A will 
be used which is more concise. These surveys are underway. 



5. A resume of employment in selected fields of agriculture, based upon 
the I960 Census of Population for Few York, has been completed. 

These data will be useful in interpreting certain aspects of our findings, 

6. A series of studies are underway throughout the state to determine the 
need for vocational and technical education. Each survey group has 
been supplied with copies of Form A and are being urged to use it in 
conducting their studies. Other information from these surveys, 
particularly regarding student interest in enrolling in courses in agri- 
culture, will be useful to us. 



7. Studies already completed have been reviewed for implications in this 
work. An example of this is the Hatch Project conducted by Professors 
Tom and Hill of our staff along with Mr, Green, a research assistant, 
(Syracuse Economic Area - Employment Opportunities in Selected 
Agricultural Occupations, 1961), 



As you can see, v/e v/ill need to pull these studies together to make 
inferences for the state. We feel that these surveys and studies will provide 
us with much meaningful data which can be used to project employment 
opportunities in agriculture for the off-farm segment, VThen added to the 
employment opportunities on the farm, we should have more realistic facts 
on which to base present or proposed programs in agricultural education. 



Second General Session - iMovember 5, 1983 

The conference reconvened at 1:16 p, m. Chairman Bail introduced 
Dr, David F, Krathwohl who directed the program for the second session, 

DESIGNIt'TG BESEAPCH EDUCATION 

David P, Krathwohl, Pesearch Coordinator, 

Bureau of Educational Pesearch, Michigan State University 

Digest of Presentation 

A, The purpose of research: A way of gaining knowledge (it is not, 
however, the only v/av). 

B, The Goals of research 

1. Prediction and Control 

2, Theory building - "v/here we want to go " 

(a) There is a lack of theory in much educational research 

(b) Theory is built by inductive reasoning 

(c) Theory building requires knowledge of variables and hov/ these 
variables interact 

(d) Theory development is essential to prediction and control 












-14- 



C, The formation of theory. 

1, May come first from inductive observation 

2, From this, one builds up hunches and hypotheses for ^testing 

3, The question then becomes, when is a theory proven r' 

(a) Often "proof" is based on past experience, opinions of 

authorities, or evidence of some sort ^ 

(b) A highly useful kind of evidence comes from a "chain of reasoning 
which is a good argument, this is what research consists of 

(c) IT IS NEVEP ABSOLUTELY PPOVEN ^ 

(d) Proof is . more a matter of intuitive or value judgement. Science 

is always testing theories , _ 

(e) A theory is accepted only when, and as long as, it escapes 

disconfirmation bv research an d exiperi mentation • . 

4, ppnpie make inductive leaps . These tend to make the 

vast advance^ rather than research itself 
D. Pesearch is building a chain of ar^mentation 
1. The chain should be as strong as possible ^ 

2 Most links are a compromise between what is realistically posable 
and an ideal that would eliminate alternative causes of the phen monon 

being observed 

3. The starting point is a hxmch or hypothesis 

4. One may contrast two methods of attack: our usual every day 
problem solving and experimentation 

5. The problem solving chain looks like this: 

(a) You must decide who you will look at 

(b) You must decide where you will look 

(c) You must decide what you will look for 

(d) You then verbally describe it and logically reason through to 

arrive at an interpretation 

6. In experimentation the same steps are present: the relation o 
your hunch or hypothesis to your observations 

(a) The who becomes the sample 

(b) The where b ecomes experimental design 

(c) The what becomes the measures , instruments, check ligts. ^ 
These are operational definitions of the variables in the hypothesis 

(d) The descriptions are numerical rather than verbal,^ and the logic 
applied to the data is statistical ; the interpretation is based on 
statistical inference 

7 Both problem solving and experimentation are chains ar^mente- 
tion which should be built so as to eliminate as many rival hypothesas 

as possible , . i.. • 

(a) Problem solving usually involves verbal descriptions, experi- 
mentation, numerical descriptions 

(b) A more precise description is obtainable through numerical 
abstraction if the measuring scales are valid 
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E. Forimilating hypotheses. 

1. These can be stated as if . . . then statements 

2. In this form the "if factor is the independent variable, "then" factor 
dependent variable 

3. A hypothesis should be based upon: 

(a) A theory or model of how a set of variables interact 

(b) A rationale which suggests some reasons for the expulation 

(c) The findings in the literature which bear on the problem 

4. A hypothesis should be simple and clear 

5. A hypothesis should be testable 

6. It is not necessary to state a hypothesis in the null form in the 
proposal, people want to know what you expect to find 

7. A hypothesis should be made before the data is in and tabulated. 
However, if the analysis suggests a new hypothesis, inform the 
reader how it was formulated, 

8. A hypotheses about what "ought to be" or "should^ be" cannot be proven, 
all research can do is to show the implications of particular value 
positions. 

F. Operational definitions, 

1. These are involved in making a hypothesis clear and testable 

2. They are behavioral descriptions of the phenomena to be observed 

3. You must tell v/hat operations you are using to measure the 
behavior, feeling, etc, 

1st example: morale. This is only observed as overt behavior. An 

operational definition explicitly tells what behavior you 
intend to observe which will be used to define high and ^ 
low m.orale. Included is a description of the manner of 
observation. 

2nd example: an intelligence test describes the behavior which is 

defined as intelligence. 

G. The Abstraction Ladder 

1. An example would start with Bess, the cow, at the lowest level of 
abstraction and go through cows, bovine, mammals, animals to 
organisms at the highest level of abstraction 

2. Research is rarely concerned with Bess 

3. Operational definitions must be used to specify the exact location oi 

the study on the abstraction ladder 

4. The abstraction ladder can be interpreted in verbal or statistical 

terms 
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H. Sampling (A means of inference from parts to the whole). 

1, Size is one aspect. Homogeneous populations .require smaller 
samples than heterogeneous populations 

2. The size of homogeneity of the sample determine sensitivity and pre- 
cision. The "correct" size of sample v/ill allow you to sensfi^a 
difference of "appropriate" size and determine effectiveness of 
treatment effect. The magic formula for determining "correct" size 
requires pilot studies or the use of other studies done previously to 
determine the variability of the phenomena one is dealing with, 

I. Sampling Plans. 

1. Random Sampling 

(a) Randomization is necessary at some stage in order to get 
probabilities 

(b) Each unit must have an equal chance of being drawn 

(c) Units must be defined 

(d) Units should relate to hypotheses 

(e) The population must be defined and is the population to which you 
wish to generalize 

2. Stratified Random Sampling 

(a) Divide one population into categories according to an independent 
variable to increase homogeneity 

(b) Because of increased homogeneity, sample size can be reduced, 
a size maintained and sample variability will be reduced 

(c) One may adjust size within each strata to variability within each 
strata. Example: in political stratification, a larger sample 
would be drawn from the Independent group than from the 
Democratic group or the Republican group because the latter 
are usually more homogeneous 

3. Area Sampling 

(a) Used in geographic studies 

(b) Reduces cost of sampling 

4. Sequential Sampling 

(a) Rarely used in educational research, but it can be used when 
resampling is easy 

(b) Take a sample, then take another and compare. Continue to 
take others until the results are within previously established 
tolerance limits 

5. Quota Sampling 

(a) Not a random sampling process therefore, one cannot, 
compute probabilities with any certainty 

(b) The characteristics of the sample are in the same proportion as 
they are in the population 

6. Panels 

(a) Again, no randomization 

(b) A panel or group of' people used as a basis for data are so chosen 
as to be representative of the population 
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J, Msasurss - Operational Definitions, These must be consist 6nt with the 
hypotheses and theory and must have: 

1. Reliability 

(a) Internal consist.ency - do the items measure the same thing 
(total score is similar to sum of score on two halves) 

(b) Stability - the instrument must give a stable reading over time 

(c) Equivalence - Forms A and B must be equivalent 

(d) Stability and Equivalence - b and c acting together 

2. Validity 

(a) Predictive: the degree to which the scores correlate with a 
criterion obtained at an appropriate future date 

(b) Concurrent: correlation of scores with criterion measured at 
the same time 

(c) Construct: demonstrates that the test acts as one would predict 
in terms of its theoretical bases 

(d) Content (face) : the test contains items that appear to be appropriate 

3. Objectivity 

(a) Persons analyzing the data substantially agree on its interpretation 

4. Objectivity and reliability are necessary for validity, but they are not 
sufficient. Validity is the most critical 



Third General Session - November 6, 1963 

Chairman: Bruce A. Gaylord Secretary: Howard Addison 

The third session of the conference was convened at 8:30 a. m. with 
Bruce Gaylord presiding. 

Dr. David Erathwohl continued the discussion of the previous 
session on Designing Research in Education. Follov/ing the pattern of ^ 
the previous day, he began by answering' questions submitted by participants. 

Digest of Presentation (continued) 

K. Experimental Design 

1. This is where - under what conditions 

2. You want a maximum treatment effect to show up over other effects 

3. The above must be done with the most efficient use of subjects ^ 

4. One can control umvanted variables (rivcil hypotheses) by masking, 
elimination, counter balancing, or statistical correction 

5. Rival hypotheses that need to be taken into consideration 

(a) History - the events which take place betv/een pre-test and post- 
test which might effect the dependent variable 

(b) Maturation - biological and psychological changes which occur 
over time 

(c) Testing - practice effect, test wiseness" 
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(d) Instrumentation - another term is instrument decay. This 

is the change in the instrument over time (bored observers, more 
expert interviewers, biased interviewers over time, etc, ) 

(e) Regression - a natural phenomenon when only the high group 
and the low group from the pre-test are used in the post -test, 

This is a function of use of extreme groups and the imperfect 
correlation between the selection measured and the dependent 
variable 

(f) Selection - the use of volunteers or some other non-representative 
group 

(g) Mortality - selective drop-out-non^respondents 

6,. Interactions of rival hypotheses 

(a) History and Treatment - a certain event may make the 
treatment "take" better 

(b) Instrumentation and Treatment - the observer may know which 
group is which and look for different things 

(c) Testing and Treatment - the pre-test may alert the students 
to what is important, therefore, they are more attentive to 
certain aspects 

(d) Selection and Treatment - the school which allows you to do 
the study may be doing so because your treatment fits their 
program. Here your treatment effect may seem greater 

(e) Reactive Arrangment - artificial arrangements and Hawthorne 
effect. The subjects feel "special" and therefore, try harder 

(f) Multiple Treatment - the first treatment may have aboncaficSal 
effect on second 

For a more complete discussion of the above rival hypotheses and for the 
designs outlined, see the section by Donald T, Campbell and Julian C, 
Stanley, "Experimental and Quasi- Experimental Design for Research on 
Teaching, " in Handbook of Research on Teaching. N, L, Gage, editor, 

Rand McNally and Company, Chicago, 1963, 

Dr, Krathwohl discussed the material included in his "Suggestions 
for the Preparation of Educational Research Proposals, " A copy of this 
checklist appears below: 

SUGGESTIONS FOR THE PREPARATION OF EDUCATIONAL RESEARCH 

MATERIALS 

David R, Krathwohl - Michigan State University 

Does your research. proposal show off its true potential? Check 
each section of your proposal against the following points — 

Problem Statement - In this section, you explain your problem and its 
significance to the field of education. 



* A checklist with special relevance for USOE Cooperative Research 
Program proposals, but also useful as a general check on any proposal. 
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1. Does your problem statement convince the reviewer of the importance 
of the proposal‘s Cite one or two concrete consequences. Indicate 
the way it builds on previous theory or contributes to new theory . 

2. Is it so written that its problem falls within the scope of the program 
of the granting agency? For the USOE Cooperative Program, are 
its implications for human learning, preferably school learning, 
made evident*^ 

3. Have you drawn a clear ring around your problem so it is clear v/hat 
you plan to include and what leave out^ Don't make your problem 
statement so brcBd you are going to solve the world's ills, 

4. Does it communicate to the intelligent professional who is not a 
specialist in your field? 

5. Is your statement so v/ritten that the hasty reader is signaled by 
topic paragraph sentences, underlining or other devices where to 
find a succinct statement of the purpose of your research? 

6. Have you briefly indicated the approach you plan to use? 

Belated Pesearch - Summarize the most pertinent related research, 
referring to the relationship between it a,nd your study in such a way that 
you demonstrate your mastery of the field and that the manner in which 
it is a new contribution is made clear. Inclusion of the theoretical base 
of the study is important. It suggests a context for the research and 
provides a rationale, 

1, This is your chance to show your scholarship. Have you carefully 
selected those materials which most directly bear on the problem? 

2, Have you summarized these in such a way that you communicate to 
the intelligent non-specialist how thej^ contribute to lay a foundation 
for your research"^ Don’t expect him to read the references amd 
don't expect him to go out and hunt them down to understand your 
proposal, 

3, Have you critically reviewed these studies and hinted at (but not 
explained here in detail) how you will avoid their flaws ? 

4, Have you included studies that are on-going and indicated liov/ you 
will mesh your study wdth these or how your study has a unique 
niche ? 

5, Studies which have a theoretical base have the greatest generality 
of use and application. If there is a theoretical base for your 
study, have you described it and clearly indicated its relation to 
the problem ? Can you show hov/ your study tests the theory or 
contributes to its extension in some wa,y ? 
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6. You can’t find literature bearing on your problem? Don't say so 
without indicating how you conducted your search, what you looked 
at, headings seaTched, etc. It sometimes helps to indicate the 
closest studies that were found, and shov/ how they fall short if it 
is not self-evident. Statements to the effect that you are starting 
de novo and nobody has ever done anything related to your study are 
viewed with suspicion by reviewers. Have you indicated how you 
ha,ve made a scholarly attempt to find your precoursers in the field? 

7. The most common error in the treatment of this section is trying 
to impress the readers with a lengthy bibliography that includes 
everything but not doing enough with each reference to show how it 
contributes. Pemember a competent graduate student can compile 
a bibliography, but only a scholar can integrate and weave the 
references into a supporting rationale for a study. 

Objectives - Here in very specific form you are asked to indicate your 
objectives, your hypotheses, the questions you intend to answer. The 
relation of this section to the problem statement must be made clear. 

This section often forms a basis for judging the rest of the proposal, so 
it is critical that the things you intend to do in Procedure are reflected 
in the hypotheses to be tested and vice-versa. 

1. Do the hypotheses or questions clearly flow from the problem 
statement or have you provided material which makes this 
relation clear? 

2. Have you flagged your obj ectives clearly so the reviewer can 
find them readily? Listing them in order of importance or po- 
tential contribution is a good idea. 

3. Have you shown their relation to underlying theory if this is • 
not self-evident to the intelligent non-specialist ? If you have 
not yet laid your theoretical base, be sure to do so here if there 

, is such a base. 

4. Do you have clean-cut specific, achieva,ble objectives that lend 
themselves to operational definitions? (You'll discuss the operations 
which define the terms of your hypotheses in the section for instru- 
mentation under Fi ocedure. ) 

5. Are these statements consistent with the Procedure section? Do 
you claim more here than your procedures will support ? Are there 
aspects of your procedure and your analysis that are not reflected 
here? 

6. Are the hypotheses and questions testable? Have you separated 
value judgements which may have crept into your treatment of 
the problem from the researchable questions which your data can 
answer? 
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7. Are the hypotheses stated in a form which indicates what you expect 
to happen? Statements in the null form make it difficult to tie the 
hypotheses to the theory and background you have developed. They 
also give an amateurish impression to the reader. 

Procedures - This tends to be one of the most carefully read parts of the 
proposal. Here the chips a,re down. In glowing terms and appealing 
generalities you have so far tried to persuade the reader that what you 
intend to do is worth while. Describing your procedural steps in detail 
brings your project down to earth in operational terms. The section 
consists of: (a) the gener0vl design, (b) population and sample, (c) data and 
instrumentation, (d) analysis, and (a) time schedule, 

(a) General design - Your design indicates how you will structure yoin* 
situation so that you can gather data which v/ill most economically 
and efficiently use your subjects and which will permit whatever 
effect you want to observe to be maximally effective, sensed and 
measured; contaminating variables either are controlled or the 
experimental effect compensated for their action. 

(1) If you have not done so previously, have you precisely indicated 
which are the variables that will produce the experimental 
effect (independent variables), and indicated equally clearly what 
the experimental effect (dependent variables) will be ? Delineate 
also the variables v/hich mxight affect the dependent variable but 
which are considered contaminating variables, (i. e. , what other 
variable might have produced the experimental results if we 

• didn't control them?) 

(2) If the independent variables are to be manipulated in experimental 
fashion, have you indicated how this will be done? If different 
levels of treatment effect will be used, indicate these and tell 

(a) why these levels were chosen, and (b) how they will be 
monitored to be sure that the levels of treatment are obtained. 

(e. g. , how did you decide to use reading books at particular 
levels of difficulty and hov7 will you know that the readers do have 
that level of difficulty?) 

(3) Are all your hypotheses accounted for in your design plans Are 
all aspects of your design plans adequately covered by the hypo- 
theses and objectives statements? 

(4) Have you indicated how you expect to control those variables 
which might otherv/ise be interpreted as ca,using the experimental 
result Indicate the m.eans of control for each of the va,riables 
and how it fits your design (randomization, counter-balancing, 
masking, etc.). 

(5) You cannot possibly control all possible contaminating factors. 
Every design is a compromise. It is important that you indicate 
the nature of the particular compromise that you have chosen, 
pointing out why you chose to control the variables you did, and 
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why you let others go. The latter is as important as the former, 
since this indicates that the lack of control of these variables are 
not an oversight, but an intentional act. Have you laid this out 
so the reader can judge the basis on which you made your compromise? 

(6) Have you avoided expediency as a reason for leaving a variable un- 
controlled unless it is judged as an unimportant variable? 

(7) If cost is a factor in the control problem, have you prepared an 
option in the budget indicating what it v/ill cost additionally to 
control potentially important factors ? 

(8) He-read your design presentation. Have you presented the most 
convincing case you can that the particular compromise you chose 
had a rational, realistic bsise that talces into account previous 
experience with flaws in design of similar research? Here are 
some frequent errors. (Not all apply to all designs, obviously). 

Did you — provide for a control grc up if one is needed? 

— take into account the Hawthorne effect? 

r- look for practice effect in testing? 

— remember to look for a possible regression effect? 

— choose your under and over achievers correctly? 

— look for factors resulting in biased assignments to 

the experimental and control group? 

— provide for cross-validation of aU multiple correlation 

results ? 

— look for potential selective loss of cases? 

— look for the effect of pre-testing subject? 

— watch for growth or maturation effects in a long-term 

study ? 

-- make sure that the control group is given all treatment 

but the experimental one ? 

The above are only suggestive. This part of the check-list could 
be as long as a book. --In fact, consult one if in doubt. 

(b) Population and sample - Describe here the group from which you 
will sample, how your subjects will be drawn and the rationale for 
the selection method. Indicate what controls by stratification or 
other me^s you will employ in sampling. Indicate the method by 
which individuals will be assigned to any groups involved in the 
study, and the rationale for the assignment method, 

(1) The description of population from v/hich you choose your sample 
is an operational statement of the group to whom you expect your 
resiilts to generalize. ^ Is it consistent with your problem state- 
ment as to the generality you have lead the reviewer to expect ? 

(2) Have you indicated the basis for stratification or other controls 
and the reasons therefore? Have you indicated the basis on 
which other factors are left uncontrolled? 



(3) Have you indicated at v/hat point, if any, in the sampling process 
random selection enters? 

(4) Have you indicated the basis for the sample size which you have 
chosen? Be sure to indicate any pilot studies which permitted 
you to estimate the size of sample needed to insure a given level 
of precision in the study. If possible, get estimates of your 
parameters from other studies and use these to indicate that 
the precision of your study will be adequa,te to show the intended 
result. 

(5) Use a sample size which is consistent with the methods you intend 
to employ. It is frequently better to employ careful methods and 

to follow-up missing cases on a small sample, than to use a large 
sample and ignore missing data. A selective factor may be 
operating in determining what is missing. 

(6) If in doubt about sample size adequacy err on the size of being too 
large rather than too small. The overhead and fixed costs on an 
experiment are such that it is less expensive to include cases in 
the original experiment than to replicate it. Indicate the basis 
for this decision in your description of the sampling plan. 

(7) Is the sampling plan consistent with the statistical model which 
you use in analysis of the data? 

(8) If you are not following a sampling plan but using a convenient 
sample, have you indicated how this is justified and how you 
expect the results to generalize ? 

(c) Data and instrumentation - Indicate the data, to be gathered and the 

collection methods to be used. 

(1) Your measures are the operational definitions of the terms used 
in your hypothesis, Are these operational definitions which are 
generally acceptable, or is there a wide gap between the measure, 
of the variable and what is intended by the usual connotation and 
denotation of the variable ? (Especially important with affective 
measures — values, attitudes, etc. ) 

(2) Have you indicated the appropriate psychometric indices 
(validity, reliability, objectivity) for the measures you are using? 
This is particularly important with nev/ or little used measures, 
Pemember you a.re writing to an intelligent non-specialist. Don't 
take it for gTanted that he knows the tests in your field unless they 
are quite common to the educational enterprise. 

(3) Have you included copies of new instruments (tests, questionnaires, 
etc.), or samples of instruments that are to be built? 



(4) Have you a measurement problem that even the top men have 
tried to solve and failed? Don't indicate that you will come up 
with an idea by and by. Indicate concretely how you expect to 
make a break-through in areas of instrument construction that 
have so far resisted such efforts. Pious intent is not enough. 

(5) Have you indicated measures of objectivity where it is important 
— rating scales, acoring of protocols, standardized intei^iews, 
essay tests, observations, etc. ? Either indicate past evidence 
or provide for its collection in the study, 

(6) Have you attended to the social-psychological aspects of data 
collection? Note control of these factors where relevant — 
control of race and sex in testing and interviewing, etc, 

(7) Indicate special provision for data, collection whore it is likely to 
involve controversy of some kind because of the topic involved, 
questions asked, clients used, etc, 

(8) Indicate how self- selection will be controlled where a number 
of school systems must be screened to find those that will 
cooperate. 

(d) Analysis - Indicate the methods that v/ill be used for each hypothesis 

or question, 

(1) Analysis must be consistent with objectives, design and sampling 
method. Use correlational techniques where you are interested ^ 
in the strength of a relation, and different.;- statistics to distinguish 
groups. A significant difference where one needs a correlation 
only indicates a correlation that is higher than zero, 

(2) In multivariate designs involving analysis of variance, indicate 
the correct error term, 

(3) Indicate how assumptions of the statistical model may be violated 
and what action may be taken if this is likely, 

(4) If special analytic tools are to be used (e, g. computers new 
factorial rotations, etc,) indicate their nature and any evidence 
of their validity and reliability if relevant. 

(e) Time schedule - Indicate in chronological order, the length of time 

required for each major aspect of the study, 

(1) This gives an indication of how carefully and realistically you 
have thought through your project. Anticipate problems and leave 
time for them, 

(2) Indicating the phasing of your project may help the funding agency 
get you started by supporting first phases even if they can't 
support the whole project. 



(3) Flow charts and diagrams are helpful in demonstrating the time 
relations and helping you to analyze and communicate the v/orking 
procedure. 

Personnel - Give name title and brief statement of pertinent experience and 
unique qualifications of personnel. 

1. Be sure to list research experience. 

2. Do not list persons without their permission, particularly persons 

of prominence, unless you specifically indicate they are to be contacted. 

3. A roster of personnel in readiness gives the impression of prior 
concern and preparation which bodes well for the project’s success. 

Facilities - Indicate special facilities and equipment which are available 
to the project. 

' l l. Actively available consultation facilities should be listed. 

2. Cooperation of schools and agencies that v/ill be involved should be 
obtained to the extent possible and evidence of their consent included. 

Budget « The budget is an operational statement in monetary terms. A care- 
fully thought-out project translates easily into monetary terms. Budget 
preparation is a good test of how carefully you have teased out the details 
of your procedure. 

1. Pemember, it is rarely true that this is your last chance at the budget. 

It can go both up and down under negotiation. 

2. You should have rationales for all items included, since these may be 
called for in negotiation of contract or gTant. 

3. It is perfectly permissible to include funds for the salaries of the 
investigators. Be sure to include fringe benefits of all personnel. 

4. Find someone who knows the costs of relevant items and salaries, 
e. g. , a purchasing agent and they '11 be able to ease your load. 

Have them check your work. 

5. Find out how to compute overhead and differential overhead. 
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Qverall points - 

!• A research proposal is basically a chain of reasoning, each step 
building on the previous one. It must hang together and be internally 
consistent. It should flow from the problem statement to the objectives. 
Your sampling and experimental procedure should match your objectives. 
Your analysis method should match your sampling plan and procedure, 
etc. Make sure that ends are not dropped, objectives slighted, data 
collected but no analysis plan, etc, 

2. You should be cognizant of the process by which your proposal will be 
judged. Have you spent extra time on the parts that will be most 
critical in the judging process? Have you skipped the items tlmt may 
be important? 

3. Have you supplied extra information regarding procedures, instru- 
ments, etc. in the appendix, out of the way of the write-up, but 
available*^ 

4. ‘ Even the best proposal writer benefits from others' critiques of his 

write-up. Has anyone read yours for you? 

5. Tiy putting the proposal aside and then coming back to it afresh and 
looking at it in perspective. 

6. Make sure your proposal falls within the scope of the funding agencies 
program. Many so-called research problems are really attempts to 
use an alreacty tested idea in a new context. Such action oriented 
proposals might better be directed to a demonstration program, or a 
development program. Are you applying to the right funding agency, 
and the right program within the agency? 

7. Proposal writing is a fine balance, On one hand you need to lay out 
a study in sufficient detail that the reviewer is convinced that you 
have a problem worth investigating and that you have the ability to 
handle it. On the other hand you need not give so much detail that 
you lay out every single possibility and eliminate any flexibility 
from your plan. Half the candy of a research project is in exploring 
an area. A researcher worth his salt will want his fun. Any proposal 
that is not built around a trivial problem cannot be anticipated in all 
details and reviewers know this. Try to find the balance line. , 

Best wishes for some significant research! 



Attention was also called to the publication, "Cooperative Hesearch 
Program, " which is available from the U. S. Department of Health, 
Education, and IVelfare upon request. 

Copies of Bulletin 15, "Sampling - Elementary Principles, " hy 
Philip J. McCarthy, were distributed to participants. Single copies of 
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this bulletin are available upon request for 25 cents each from Distribution 
Center, New York State Sbhool of Industrial and Labor Pelations, Cornell 
University, Ithaca, New York. 

An expression of appreciation was given to Dr. Krathwohl for his 
contributions to the conference. 



Fourth General Session - November 6. 1963 

The session w 3 called to order at 1:30 p. m. by Bruce Gaylord, 

Dr. Harold B. Cushman, Conference Chairman, gave instructions 
to the afternoon work groups. These instructions were: 

1. Study reports of completed research projects or research proposals 
submitted in advance by several states. 

2. Suggest improvements in the design of the research projects or pro- 
posal studied and prepare a committee report. 

3. Discuss research proposals of committee members. 

The members of the various v/ork cfrouns and research topics studied 
are listed below: 



Group I 

"Testing A Procedure for the Improvement of the Beading Ability of Pupils . 
in Vo-Ag Classes" by V. B. Cardozier 
Chairman: Daniel Koble 

Members: Cushman, Hash, McClay, Penzelman, Pearce 

Group II 

"The Effectiveness of Teaching Parliamentary Procedure Through Use of 
Programmed Instruction” by James Eanneman 
Chairman: Homer Judge 
Members: Barwick, Annis, Drake, Lechner 

Group III 

"The Pennsylvania Vo-Ag Interest Inventory" by B. W. V/alker, G, Z, 
StevemSy and N. K. Hoover 
Chairman: Gene Love 

Members: Bail, Becket, Campbell, Bobertson, Taylor 

Group IV 

"A Study to Identify the Nature of the Major Professional Difficulties En- 
countered by Teachers of Vo-Ag in Texas" by Earl S. Webb 
Chairman: Harold Noalces 
Members: Stevens, Gaylord, Gilman, Wolff 
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Group V 

"The Comparative Effectiveness of Two T 5 T®s of Organizing and Teaching 
0 Farm Credit to Vo-Ag Classes" toy David McClay and Otto Legg 
Chairman: Virgil Christensen 
Members: Addison, McGhee, Katz, Nielsen, Towne 



Fifl-h Gf^npral Session - November 7. 1963 

Chairman: David B. McClay Secretary: William E. Drake 

The fifth session was convened at 8:30 a. m. by David F, McClay, 
presiding officer. 

This session was devoted to critiques of research reports and pro- 
posals studied by the work groups during the previous afternoon, 

A panel of four researchers listened to the reports by the group 
chairmen and reacted to the strengths and v/eaknesses of the committee 
reports and the research being criticized. The panel members were: 

Jason Millman, Assistant Professor, Educational Psychology and 
Measurement, Cornell University 

Duane M, Nielsen, Pesearch and Teacher Education Specialist, U, S. 

Office of Education 

Glenn Z, Stevens, Professor, Agricultural Education, Pennsylvania 
State University 

Lawrence B, Darrah, Professor, Marketing and Agricultural Economics, 
Cornell University 

Work Group Committee PeTX)rts 

The material presented herein includes only the committee reports 
with implications for improving the research studied. Copies of the research 
proposals were distributed and a limited number are still available from the 
Agricultural Education Division at Cornell, 

Group I Chpinmnn - Daniel Koble, State Director, Agricultural Education, 
Delaware, "Testing* A Procedure for the Improvement of the Beading 
Ability of Pupils in Vo-Ag Classes, " 

Overall, the committee thought this was a reasonable research pro- 
posal, Many strong points could be listed. However, in order to improve 
this proposal, the committee suggests the following changes: ^ 

1, Cite evidence to support statements made under justification. 
(Assumptions made would be best reported as hypotheses). 

2, Make a more comprehensive review of literature in related 
disciplines. 
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3. Weave review of literature together. See that it will serve as 
a measure of progress achieved by previous research. 

4. Construct a more specific objective, e. g. , to test the effective- 
ness of a. proposed program for improving the reading compre- 
hension of 9th and 10th grade vocational agriculture students in 
selected Maryland high schools. 

5. Transfer present objectives not covered in revised objectives to 
procedure. 

6. Specify procedure for selection of sample. (Spell out criteria, 
size, etc. , in greater detail). 

7. List statistical procedures that v/ill be employed in analysis of 
data. 

Group II Chairman - Homer Judge, Head, Agricultural Education De- 
partment, University of Massachusetts. ’'The Effectiveness of Teaching 
Parliamentary Procedure Through Use of Programmed Instruction. " 

Statement of the Problem 

1. Should be in statement form rather than question form. 

2. In applying Krathwohl's criteria, Hanneman's introduction 
should be included in the statement of the problem. 

3. The topic treated is not in keeping with the majority of the vo- 
cational areas taught in vocational agriculture. The committee 
feels that a programmed experiment in a non-academic area of 
vocational agriculture would be more effective in providing 
the evidence necessary for agricultural educators to malce 
valid decisions regarding the effectiveness of programmed 
instruction in vocational agriculture. 

Use of Previous Research and Other Auxiliary Information 

1. Build foundation for generalizations through a more compre- 
hensive reviev/ of previous research. Cite specific, concrete 
examples. ^ 

Purposes 

1. IViakesno explanation of any other kind of instruction other than 
programmed instruction, if any. The committee believes that 
the purpose is to make a comparison rather than to determine 
an effective method. Wliat is programmed learning being 
compared to ? 

Population Selection 

1. Would be much better if he stated the control groups were made 
up of practically all the Ag Departments in South Dakota other 
than experimental groups. 
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Sampling Techniques 

1, Appropriate random sample for size of state, 

2, Could be more explicit of procedure used in random sampling. 

Assumptions and Limitations 

1, There may be a Hawthorne effect, 

2, Differences between the experimental groups and control will 
be due to subject ma,tter covered by program not to pattern of 
teaching, 

3, He hopes a positive correlation v/ill exist between the success 
on the criteria examination and the ability to use the information 
during an active meeting. This is an unwarranted assumption, 

4, If his test is used in South DaJkota, he doesn’t have a limitation, 

5,. One of the greatest limitations of the study which was recog- 
nized by the researcher was the lack of external criterion for 
determining the validity of the test used, 

6, Missed a basic assumption, that the knowledge of parliamentary 
procedure by the Ag III and IV groups is a just criteria to 
measure the knowledge of Ag I students. 

Variables Considered 

1, Variables used in this test are programming and nothing, 

2, Example - (Demonstrating the use of fertilizer on one plot and 
no fertilizer on another). 

Criterion Used 

1, Does criterion test evaluate the program, the method, or both? 
Should someone else have made out the criterion examination? 

Model or Experimental Design Used 

1, Using acceptable design but not applied in a manner which 
will bring forth any proof. Demonstrating a procedure 
rather than gaining a comparative measure against conven- 
tional teaching methods. 

Data Collection, Procedures Follov/ed, Treatment of Statistical Data 

1, Data collection acceptable. 

2, Used an apparently acceptable procedure for determining the 
reliability of the criterion examination, 

3, The committee agrees that acceptable statistical methods are 
being used. However, the committee questions whether or not 
the results of this statistical analysis will offer any evidence 
that is not readily available by observation, 

Peporting, Summarizing, Conclusions and Expected Utilizations of 
Pesults^ 

Not included in this propos?^!. 



"Demonstrating the Teaching of Parliamentary Procedure 
Through the Use of Kfogrammed Instruction" is suggested as 
a more appropriate title for the study as seen by the committee. 



1 . 

Title 

1 . 
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other PocommencMions 

1. This is a much better study than the criticism of the com- 
mittee would lead you to believe. 

Group III Chairman - Gene M, Love, -Assistant Professor, Department 
of Agricultural Education, Pennsylvania State University, '‘The Penn- 
sylvania Vo-Ag Interest Inventory. 

Generally spealdng the review committee found very few criticisms of 
the study titled "Development of a Vocational Agriculture Interest Inventory 
for Guidance of Eighth Grade Students, " conducted by Dr. Paul Walker 
at the Pennsylvania State University. It was the unaminous opinion of the 
group that the study had been v/ell formulated and executed. The purposes 
were sound and respectable. 

The one criticism which the committee was able to make concerned 
sampling technique. Although the sample v/as large enough to permit the 
researcher to accomplish his objectives and to generalize to a point which 
V 70 uld permit the test to be used with Pennsylvania 8th grade students 
where 9th grade vocational agriculture is offered, the committee felt 
that a broader norm base or a more complete sampling of a wider range 
of 8th grade students from other areas of the state may have permitted 
the researcher to generalize to an even further degree. For example, 
will the Pennsylvania Inventory predict far success in agriculture classes 
in a New York or Philadelphia school where agriculture is taught? 

Other questions asked by the committee were: 

1. Does the Pennsylvania Vo-Ag Interest Inventory discriminate 
between students with high and low IQ’s ? 

2. V\/hy wasn’t the Kuder-County Agent, Veternarian and Forsterer 
Keys correlated with the Kuder Vo-Ag Key and the Pennsylvania 
Vo-Ag Interest Inventory ? 

3. Could this study have been expanded or should it be follov/ed 
up with additional research to determine the value of the 
Pennsylvania Interest Inventory ? 

4. Should the response items have been patterned more closely 
after the Kuder v/hich used three item choices for predicting 
success at higher grades ? 

Statements: 

1. Test is limited to one interest group — agriculture. 

2. Vo-Ag Key for the Kunder Preference Inventory v/as a sig** 
nificant development. 
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Group IV Chairman - Harold L, Noakes, Associate in Agricultural 
Education, State Department of Education, New York. '|A Study to 
Identify the Nature of the Major Professional Difficulties Encountered 
hy Teachers of Vo-Ag in Texas. *' 

1. Descriptive research should suffice to base further research 
upon. 

2. One-way tables do not provide data to the degree that two-v/ay 
tables will. 

3. Multiplying responses of ten supervisors in the state by the 
number of teachers in their respective area and reporting 
responses in terms of weighting. 

4. Train supervisors to interview on basis of random sample from 
each district to provide greater response and completion. 

5. Cut down on items to obtain key items or indicative items for 
an entire area and further guarantee complete responses on the 
instrument. 

6. Prepare a table of contents and number pages. 

7. Preparation of “dummy” tables or data tables in which to enter 
data upon receiving instruments from respondents. 

8. Describe procedure for identifying separate items of the instru- 
ments. 

9. Indication of review of literature in the area of study. 

10. Improve mechanics of the instrument to prevent incorrect or 
misinterpreted responses in headings provided. 

11. Instrument calls for negative and positive response but tabulation 
is only provided for the negative reason within the es^erience. 
Two-way tables would prevent this difficulty. 

12. Table describing alma mater of respondents is not relevant to 
the study. 

13. Clear, concise, focused purposes were commendable with the 
exception of definition of terms: 

(a) Professional 

(b) Nature 

(c) Difficulties 

14. Becommend the description of procedure for follow-up on the 
non-respondents. 
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15. Details of design appear to be lacking. 

18. Provisional comments fail to indicate the need foi the study. 

17. Failed to answer PUPPOSES v2, 3, 4, and 5 because the pro- 
fessional difficulties are not reported in terms of the various 
criteria identified in these purposes. 

18. The study should be in two parts on the basis of its purposes, 
and not on the rationale of the sample ! 

19. Need for delineating the study in terms of assumptions. 

20. Potential exists for skev/ed results due to the absence of graduates 
of some institutions of higher education in Texas, 

21. Total population for sampling procedure is not identified. 

22. Possibility of bias exists as a result of sampling procedure, 

23. Data not analyzed with respect to purposes cited. 

Group V Chairman - Virgil E. Christensen, Assistant Professor, 
Agricultural Education Division, Cornell University. -‘The Compa,rative 
Effectiveness of Tv/o Types of Organizing a,nd Teaching Farm Credit 
to Vo-Ag Classes. " 

The Problem 

1. The statement of the problem lacks in "robust-tivity" Bs does 



the rest of the proposal. 

(a) Too heavj^ a reliance on Legg’s objectives instead of stating 
objectives of this study. 

(b) Ambiguity of terminology in points 1 and 2 under objective A. 

(c) Part B fails to identify "commercial literature. " 

(d) Hypotheses are never stated in the proposal, 

(e) The variafoles are not clearly defined, such as what tjpes 
of questions were to be asked in the achievement tests, 
what were the dependent and independent variables. 

(f) By referring to the summarjj^, it was possible to determine 
that different answer sheets were used in testing. No 
mention of such use of different answer sheets was made in 
the statement of the problem. 

(g) It does not appear that hypotheses v/ere logically deduced 
from some theory or reviev; of literature. 
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Instrument Developed 

1, We have no way of knowing what instrument was to be used or 
how one was to be developed, (F, 2 of #61-240), 

2, No evidence of a pilot program or other means of establishing 
reliability, validity and objectivity of ’‘the instrument, " 

(P, 2 of #61-240), 

3, No indication of the nature of the questions to be used in test 
and retest — such as fact and figure, general principles, 
problem solving, etc, 

4, No indication of whether retention test was to measure recall, 
recognition, reconstruction, or relearning, 

5, Two control measures were included but additional but equally 
important ones might have been included such as mathematical 
competency. 

The Desicm 

1, Population studied was not clearly defined nor was. sampling 
procedure outlined, 

2, No control was listed in the design and the model is not very 
explicit. 



X States Y States 

9 and 10 L P 

11 and 12 F , L 

Adults L P 

Adults C C . 



(2nd page of #62-276) 

3, No way of laiowing on what basis size of sample was delimited. 

4, We have to recognize the vested interests of the supporting 
agency but a great sacrifice of representativeness was thereby 
lost. ,A more tightly designed study in one state would probably 
have yielded more to both the researcher and sponsor. 

Proc edure 

The treatments and methods of collecting data were not described 
so that an independent investigator or reviewer could replicate 
or evaluate the study. 

The terminology inadequately describes the characteristics of 
the sample, (i, e, , F, 1 of #62-276), 

No directions were given or were plans for developing directions 
on the use of instruments included in the proposal. 

Copies of the instruments were, not included as appendages to 
the proposal. 

Classification or grouping systems were not explained. 
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2 . 

3. 

4. 

5. 



Analysis 



1, In addition to previous statements about the instruments, no 
statements concerning statistical assumptions or techniques 
were made in the proposal, 

2, The 1% level of significance, as evidenced by the summary, 
seems appropriate for this study. 

Miscellaneous 

1, P, 3, v61-240 ~ The use of the term "or variations thereof" 
creates considerable uncertainty as to study procedures, 

2, P, 3, TP 3, - The time schedule is not covered by probable 
duration of three years, 

3, The lack of commonality of instructional conditions may create 
control problems, 

4, Operational definitions would need to be listed, (Example - 
Title), 

5, The citing of literature must be pertinent and develop a tight 
chain of reasoning, 

6, Without the instruments attached - we cannot tell whether 
control ivas given to important variables. 



Sixth General Session - November 7, 1963 

The final session of the Research Conference v/as called to order 
at 1:15 p, m, by David P, McClay, 

PESEAPCH EXPERIENCES 

Lav/rence B, Darrah, Professor 

Marketing and Agricultural Economics, Cornell University 

More than once during the past 2 l/'2 days you have been given, as 
a part of this program, a man-sized portion of research techniques and 
philosophy which has become commonly Imown as "methodology?’. " 
Undoubtedly, more than once you have become enthusiastic, enchanted, 
enamoured, even enraptured and perhaps even infatuated with research 
procedures. Unless I am seriously in error, chances are good that before 
long some of this will wear off and you will becoiie more realistic in jt’out 
attitudes, less critical of other people's research, and more confused re- 
garding’ v/hat to do and how to do it -- because all of you will be v/orking 
within tight limits imposed by lack of funds, manpower, and other University 
duties. 



I have drav/n as my text for the afternoon lesson, from a statement 
made by Dr, Heady of lov/a. State at the 1961 International Conference of 
Agricultural Economists, the following quotation v/hich reflects my thoughts 
on research methodolocy as an end in itself, I quote: 
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”Most fields of science have their particular fads and attempts at^ 
keeping in style, This fact has been emphasized by Pogowi/in his classi- 
fication of terminology in the broad fields of social science, as well as in 
political science specifically. Following an earlier article on linguistic 
class indicators in present-day English', he has classified terminology 
(and, presumably, activity associated with terms) into U (upper class) and 
non-U, The term U was originally used to distinguish words in 'correct, 
proper, legitimate, appropriate usages' from those which are 'incorrect, 
not proper, not legitimate'. Hence, Pogow classified scientific terms 
which are 'sophisticated, modern, knowledgeable', as compared to those 
v/hich are 'crude, obsolete and uninformed', using the distinctions of U and 
non-U, Part of his classification follows: 

'Currently, behavior and methodology are entrenched in U 
vocabularly. Symbol is U; word is non-U. Variable and inter- 
action are U, So are model, especially equilibrium model, 
matrix, and cells (cells in matrix are exceedingly U), Empirical 
is U , , , whereas value is non-U, Quantitative is U, against quali- 
tative hon-U, Mathematics and statistics are U , , , Game theory 
is ultra-U in U circle , , , The New York Times is U; all other 
newspapers are non-U , , , U departments have chairmen, non-U 
departments have' heads . , , Pesearch is U , , , Teaching and 
teaching assistant are non-U , , , Pelated terms are U scholar, 
and non-U administrator , , , U professors go on leaves; non-U 
professors take sabbaticals , , , Pesearch books and articles are 
U, textbooks are non-U, and collections of readings are the most 
ncu-U of all , , , '” 

There are two types of research. There is research for the sake 
of research, and, most important, research to solve problems. The 
latter is where your emphasis should be placed. To conduct successful 
problem-solving research requires three steps: 

1, The visualization and definition of problems that need to be solved, 

2, Obtaining accurate data and maldng trustworthy observations, 

3, Using the proper ''tools" to carefully analyze the data. 

Most researchers do not know all there is to know in their field, so 
good problems usually can be visualized at a more rapid rate than funds and 
time available permit to be solved. 

The job of a research worker is to recognize the important problems 
and use the most appropriate and effective tools available to solve these 
problems. However, recognition of the problems, in themselves, is not 



\J Pogow, A, A, , "A Short Note on U and non-U in Political Science, " 
Western Political Quarterly . Vol, }QII, No, 4, 
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oufficisnt, Th© probloms must b© cl©cirly d©fin©d. This is oft©n ov©r— 
lobk©d b©caus© (1) it is a dull, routin© proc©ss, (2) th© tim© is usually 
short, (3) it is fr©qu©ntly and ©rron©ously assurn©d that bias and pr©judic© 
will b© inj©ct©d into th© study, and (4) it is larq©ly a non-U activily. 
Through d©finition of th© probl©ms, a r©s©arch v/ork©r plac©s hims©lf in 
a position to pr©par© qu©stionnair©s that will avoid th© coll©ction of infor- 
mation of littl© or no valu©. Not only will this sav© tim© and mon©y but it 
will insur© that th© data coll©ct©d will b© p©rtin©nt to th© probl©ms. Far 
too many res©arch work©rs in all fi©lds have b©on grabbing hold of*an 
apparently pr©stigious tool and th©n trying to find a probl©m or som© data 
on which to us© th© tool. Th©r© is no d©nying that w© n©©d n©w tools with 
which to v/ork. Th©r© is no denying that w© need to make better us© of 
th© tools we now have. But the real problem is whether the method or tool 
is usexul in solving the problems. All coo often we, in the social sciences, 
rely upon surveys to obtain information when there may be alternative ways 
01 ootaining better , more reliable information. In my ov/n area of work, 
we have made use of, for example, the photo-projection technique to obtain 
survey data. In this procedure a photo depicting a particular family situa- 
tion is presented to the respondents and the questioning is related to what 
the respondent believes the situation represents and how and why the 
respondent believes the family depicted v/ill react to a given condition, 
v7e have used the relatively new psychological procedure of asking respon- 
dents v/hattheir neighbors v/ill do or have done under given conditions and 
the reasons for such, ^ Both of these procedures are valuable means of 
obtaining data under circumstances wherein the true answers may not be 
socially acceptable or rational. Also, we have resorted more and more to 
the use of controlled marketing experiments to discover consumers' 
reactions rather than use surveys, V/e have found that there is a major 
difference between what people say they will do and what they will actually 
do when there is a price tag involved, or if the true ansv/ers lie in the area 
that many consider to be socially unacceptable or irrational. 



Gathering of accurate data and mailing trustworthy observations is 
one of the most non-U activities of all, yet it is absolutely essential for 
good research work. Faulty data may rnalce research v/ork worthless or, 
even worse, misleading, I^eep in mind that the greatest help in getting 
the correct data and the proper answer is to ask the right questions or to 
use the appropriate setting from which the right answers may be obtained. 
Even asking the right questions can lead to errors because people v/ill 
often mislead enumerators through an effort to cfive the kind of ansv/ers in 
which they believe the enumerators are iEterested, This suggests that one 
may need to run a check to measure the degrree of falsification or use a 
different study procedure that would minimize the potential error. For 
example, last February we conducted a survey of consumer familiarity v/ith 
and use of 21 new food products. As a check, v/e included as one item a 
fictitious product with a familiar company name, and one fictitious company 
name. For the fictitious product with a familiar company name, about five 
per cent of the housev/ives said they v/ere familiar with it and a third of 
this group said they had made purchases of it. For the fictitious product 
with a fictitious company name, three per cent reported knowing of it and a 
fourth of the group claimed to have purcliased it. Of course, some of this 
reported lamiliarity and use could have been honest errors resulting from 
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confusing the product with soire other one, but the moral of the story still 
remains. — use the best research techniques available to avoid expected bias 
because even asking the right questions may lead to problems. 

Perhaps this argues for the need to identify the answers, as closely 
as possible, prior to conducting a survey to insure asking the respondents 
the right questions. Let me give you a couple illustrations. A survey re- 
garding consumers' acceptance of v/axed milk cartons encountered an un- 
expected low acceptance rate on the part of the Jewish people surveyed. 

The problem, which was not anticipated, was that these people feared the 
wax coating may have been derived from animal by-products, Ead this 
situation been anticipated, alternative questions on religious beliefs and 
the impact of such could have been asked, or a statement made concerning 
the derivation of the wax, either of which should have resulted in more 
appropriate information. As a result of another survey, a firm produced 
a sanitary product wrapped so it could be carried by potential users and 
used for those emergency situations that do develop. Sales never re^hed 
the anticipated level because customers bought the item once and carried 
it just for an emergency - and apparently emergencies did not often arise. 
Clearly, asking questions regarding the frequency with which emergencies 
arise would have resulted in a more realistic appraisal of the potential 
market. 

Asking the right questions in the right manner is difficult to do, 
and especially if the true ansv/er is in the socially unacceptable or 
irrational area - as many of the answers undoubtedly are in your field. 

Even the relatively new photo -project ion technique, or the procedure of 
asking a person why her neighbors do or do not follow a particular practice 
faces some of the same handicaps as direct surveys when the answers may 
not be socially acceptable. Even asking a fev/ appropriate questions in 
advance of a study may well save considerable work as is illustrated by 
a consumer acceptance study we made of a neiv type egg carton (describe 
carton). Sales started out relatively high and dropped throughout the 
market test. The basic problem was the consumers could not figure out 
how to open the carton. Some used knives, screwdrivers, forks, and 
what have you to open the carton — never stopping to read the simple in- 
structions that said "open here, " Asking a number of consumers just 
one simple question, the right question, would have turned up the true 
answer promptly and saved the company several thousands of dollars in 
production and market testing expenses. 

Asking the right questions requires a solid definition and under- 
standing of the problem with which you are to work. This, as mentioned 
previously, is an extremely non-U activity but mark it down, underline and 
footnote it — because it is fundamental in good research. 

Now for a look at some sampling patterns that have been used. 

First, let's note a case where a sampling pattern may be cleverly designed 
but stEl be a "trap" for the researcher. Take a cholesterol study recently 
made in the Boston area. A sample of 1, 000 first goneralion Irishmen was 
selected for study in the area. Then an attempt was made to get blood 
brothers of these folks still living in Ireland, and 300 were obtained. So 
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far, this sounds like a U-sarnple. Analysis of the two groups showed that 
the ones in Ireland, and note this carefully, ate substantially more fat, ate 
more calories, were lighter weight and had a lower incidence of heart 
trouble than their brothers that had immigrated to the United States. This 
analysis surely indicates some interesting results -- something happened 
to the U. Do you see any pitfalls in the sampling pattern? (Allow discussion. ) 

Let*s look at it critically. First, those that picked up and moved 
3, 000 miles to the 'land of opportunity” mu,st have been different than those 
that chose to remain at home — they must have had more get-up-and-go, 
must have been unsatisfied ivith their conditions and willing to gamble for 
a better life. And this is right. In this country' the people v/ere largely in 
executive type, high-pressure jobs; those in Ireland, were primarily 
farmers and laborers. This suggests that even with a clever, clearly U 
sampling idea, things can backfire. This is one of the many pitfalls into 
which research workers may stumble unless they ask the right questions 
about their populations and keep their eyes open at all times ! It certainly 
would have contributed to the study to do the reverse, i. e. , talce a sample 
of people here and corapare them v/ith brothers who had moved to Ireland 
to live, but I suppose this is a non-U activity. 

In another study where sampling w*as used, a normally very minor 
and completely unsuspected factor led to a costly production and marketing 
error. One of the major toilet paper manufacturers in the country decided 
that they had better get into the production of colored paper --to keep pace 
with competitors or to give greater consumer satisfaction, I don't know 
which. They hired one of the nation's leading research firms to determine 
what colors they should produce. A probability sample was drawn, a 
survey made, and the colors one would expect to be preferred — green, 
yellow, blue and pink shov/ed up, but topping the list was rose. The 
company made rose colored paper but it didn't sell. Now, you might 

ask, v/hat the went wrong. Did people change their minds ? Do 

color preferences for toilet paper change with the seasons ? Do people 
want cool colors in the summer and warm ones in the winter Was it a 
poor sample? or V/hat (Allow discussion). The ansv/er was in 
sampling, although it was a good U-sample from all apparent aspects. 

About a third of the families interviewed lived in a major housing area 
in which all the bathrooms v/ere rose colored and they wanted matching 
paper. For the country, there were relatively few rose colored bathrooms. 
But, how could the sampling specialists ever knov/ this in advance. I am 
sure that checking the color of bathrooms is an extremely non-U activity 
for research workers. I might add at this point, that a controlled market 
test in v/hic h the various colors were offered for sale to consumers would 
have given a more reliable answer. 

Sometimes emergencies arise and you need information real quick. 

Here sampling is of real value because you may not have the time and 
money to make a complete enumeration, or are too lazy to do it. In 
Pochester last fall there was a major egg merchandising study underv/ay 
and we were cooperating in a limited way with the program. Very early, 
we found a need to know, currently, the relative sales volume of the new 
eggs. Since the eggs being sold came from a number of suppliers, v/e 
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could not get the needed data promptly, if at all. Likewise, the stores would 
not permit us to keep sales records. What v/ould you do? (Allow discussion) 
We simply walked into each store once on Monday and Tuesday, twice on 
Wednesday and Saturday, and three times on Thursday and Friday, picked 
out a cart, put an item in it and "shopped" through the store counting the 
dozens and brands of eggs in each shopping cart. Of course,- we usiially 
replaced the item or items we were presumably going to purchase on the 
shelf, parked the empty cart and walked out of the store. Snooping in other 
peoples' shopping carts is a non-U job, but we came up with a figure of ten 
per cent of the egg sales being the new item — which was exactly what the 
final figure was when it became available after the test was over and the 
dust had settled! 

Another survey technique we have used is the Western Union Survey 
system. For at least one reason, this is a U technique. I was amazed at 
the prompt attention and cooperation of a respondent when the operators 
announced over the phone that "This is the Western Union Survey Operator. " 
Not only were all questions answered promptly but we even had many re- 
spondents go to the refrigerator to note the brand of a particular product. 

I suggest this survey method where time and funds are limited and the 
questionnaire is relatively short and simple. 

A random sample is a fine tool to use in research work. But, 
whenever and wherever possible, support it with all the information you 
have available, because a random sample may not be a representative 
sample. Again, may I illustrate a situation in which good judgement made 
it possible to make a random sample more of a representative sample. 

We often study consumers in Syracuse and have come to know the City 
pretty well. In one study, we stratified the city into four income areas 
based on U. S. Census data. Within each area, we numbered each of 
the. blocks in a serpentine manner. From a table of random numbers we 
selected X number of blocks in each area for study. After this, the side 
of the block (N. , E, , S. , W. ) for study was determined (from a table of 
random numbers) and lilcewise the starting point on the side, i. e. , the 
first, second or third house. It was an ideal random sample from the 
office standpoint. But in a practical sense it was terrible for the low 
income strata. A very high proportion of the low income blocks fell in a 
run-down swampy industrial section in which practically no oiie lived 
while a major low income section was virtually missed. V/hat would you 
do? (Allow discussion). We simply replaced the bulk of the blocks in 
this ^ea with randomly selected blocks in the other low income sections. 
Here, judgement took over, apparently a non-U activity, and I am sure v/e 
ended up with a more representative sample than the original one, I expect 
any statistician, with the same knowledge of the area, would have done the 
same thing. 

Briefly, then, let me summarize my random comments by em- 
phasizing again (1) the real neod to design research projects to solve 
problems, (2) if a survey is required, develop your questionnaire carefully 
and be sure that you are asking the right questions, (3) consider the possi- 
bility of using other techniques or controlled experiments wherever possible 
as they offer a means of minimizing errors in areas where the true answers 
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may be socially unacceptable or irrational, and (4) don’t forget to m^e 
maximum use of your major God-given asset — good judgement — in 
planning and conducting your research work and in analyzing your dat^ 



OUB NATIOFiAL CEMTEP FOB ADVANCED STUDY AND BESEABCE 

m AGBICULTUBAL EDUCATION 



c. W. Hill, Professor, Agricultural Education, Cornell University 

and i.iember of the Board of Trustees 

An article in the March 1983 issue of The Aaricultural Education 
Magazine by Bobert Taylor entitled, "The National Center - A New 
Besource for the Profession" gives the development of the Center, its 
purposes, how it operates and the nature and role of the Center. You have 
read the report. However, I would like to point out a few items. 

1. The idea of a national center came into sharp focus in 1959. 

The Center was established in 1962 at Ohio State University. 

2. The purpose of the Center as stated by Taylor in above-mentioned 
article; "It is envisioned that the Center will supplement and 
enliance the existing graduate and in-service programs of the 

various states. Its role is to strengthen, not duplicate, existing 
services. " 

"The broad purpose of the National Center is to develop competent 
personnel who can provide leadership in further developing agricultural 
education. Some of the major concerns are: 

1. "To provide continuing reappraisal of the role and function of 
agricultural education in the public schools; 

2. "To upgrade state leadership in agricultural education through 
advanced study and in-service education programs; 

3. "To provide educational opportunities for individuals contem- 
plating foreign assignments and leaders from other countries 
responsible for administration and supervision of agricultoal 
education; 

4. "To coordinate research und.jrtalcings among states, initiate 
research where needed, and strengthen state programs of 
research. " 

One of the Center's first undertakings was a workshop for state 
supervisors which was conducted in Tennessee in January, 1963. This was 
in cooperation v/ith state departments of vocational education in Alabama, 
Mississippi and Tennessee. It is the purpose and function of the Center to 
conduct workshops and conferences in regions or areas away from Ohio 
State. One of the next undertalcings was a tiiree-day Besearch Coordination 
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Conference on Agricultural Occupations Studies the last of May, 1963. 
There were 25 participants from 14 states and the District of Columbia, 

The major undertaking was the National Seminar entitled, “A Design for 
the Future" conducted at Ohio State, July 22 to August 2. There were 
104 state and national leaders, representing 34 sta,tes and several 
foreign countries. The report of the seminar is now at the binder's and 
will be mailed to the states in a few days. The Advisory Committee 
recommended that the r^orts be sold to states and individuals. 

The major problem in getting the Center operating as envisioned 
is the need for money, Ohio State, the State of Ohio and vocational funds 
allotted to Ohio have paid esipenses to date. Contacts have been made to 
secure grants from Kellogg Foundation, Sears Foundation, Danforth 
Foundation and others, Recently, conferences have been held to make 
known the need and obtain money from vocational education funds. 

Dr, Taylor and others are vigorously pursuing and exerting great effort 
to obtain finances needed to make the Center function as anticipated. 

This will require a sizeable sum. 

Programs' to be sponsored and conducted by the Center in the 
coming year are: 

1, Conference on Developing Guidelines for Experimental Programs 
and Summarization Procedures for Agricultural Occupation 
Studies, January 13-15, 1964. 

2, A National Seminar on Agricultural Technician Training, 
Scheduled for late spring or summer, 

3, Summarization Conference on Agricultural Occupations Studies, 
Date will depend upon completion dates of a number of state 
studies. 

4, Teacher Education Conference. A national invitational 
conference, 

5, Research Project: An Inventory of State Vocational Education 
(All Services) Leadership, Including a Survey of Selected 
Aspects of State Personnel Iblicies, 

The AVA Research Committee and the Vocational Division of the 
U, S, O. E, are co-sponsoring a National Vocational Education Research 
Conference on the Administration of Research. This conference will 
be held at Ohio State University, March 15-21, 196^ with Dr. Robert 
Taylor acting as chairman and leader for this conference. This will be 
an invitational conference for State Directors and Supervisors for the 
most part. 

The Center will serve in ways in addition to the specific purposes, 

A few of these are: 






:iii1CC55;i-^.33=*5^ 




-43- 

1, It is free. Participants are free to explore and look into the 
future v/ithout bein,^. bound bjr present programs and regulations, 

2, Already there is a "halo effect-' assocM§d Mthjhe Center, 

; . . ■ . Specifically, individuals in agricultural education are more 

willmg and ready to explore and accept changes, ^ For example, 
supervisors look more favor-ably toward advanced study and 
research. ' 

3, The Center c^n bedikened to a catalytic agent in program ex- 
ploration (research), planning and development, 

Homer Edwards and myself are your representatives on the AVA 
Advisory Committee, So far, we have met twice a year. The next meeting 
is scheduled for December 8, 1963 in Atlantic City, New Jersey, It is our 
responsibility to serve you. So, it is requested that you let us know your 
thinking as to what the Center should do and how it should function, 

PESEAPCH IMPLICATIONS OF THE STUDY OF EMPLOYMENT 
OPPOPTUNITIES AND NEEDED COIVIPETENCIES IN NON-FAPM 

AGPICULTUPAL OCCUPATIONS 

Duane M, Nielsen, Pesearch and Teacher Education Specialist 

U, S, Office of Education 

Summary of Pernarks 

Dr, Nielsen made an informal presentation and used the overhead 
projector to emphasize important points. He pointed out that the most 
recent legislation dealing v/ith vocational education has been a result of 
the report of the Panel of Consultants on Vocational Education, 

The implication, of the National Education Improvement Act of 
1963 was discussed. Dr, Nielsen stated that H, P, 4955 was a result of 
the larger act. 

To be fully informed. Dr, Nielsen urged each member of the con- 
ference to obtain the following documents: 

1, HP 4955 - Senate version passed on October 8, 1963, 

2, Senate Peport Calendar #531, 

3, Congressional Pecord of August 6th, 

4, Congressional Pecords of October 7 and 8, 

Glenn Stevens discussed contemplated changes in the publication 
Summaries of Studies in Agricultural Education. He mentioned it would n 
be impossible to continue having it printed in Washington in its present 
form. The possibility of merging Agricultural Education research 
reports with the other vocational services for a new publication was 
brought up. 







I Dr. Stevens distributed the following materials: 



1. You and Besearch -• AVA. 

2. Abstracts of Research Studies in Agricultural Education Com 
pleted in 1962-63 in the North Atlantic Region. 

3. The Statistical Table - Penn State. 

4. Research Design in Agricultural Education, by Glenn Stevens 
The conference was adjourned at 4:30 p.m. 
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